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« METHOD AND DEVICE FOR MODIFYING DATA IN AN ENCODED DATA STREAM » 



FIELD OF THE INVENTION 

The present invention relates to a method of modifying data in an encoded 
data stream corresponding to successive pictures divided into sub-picture, comprising 
the steps of : 

- decoding said encoded data stream ; 
5 - - re-encoding the decoded data stream. 

The invention also relates to a video processing device for carrying out said method. This 
invention, for instance useful when a broadcaster wants to introduce additional data such 
as its own logo into a sequence of pictures, finds applications not only in the field of 
MPEG-2 compression (MPEG-2 will be the common standard for TV broadcasting), but 
10 more generally in any digital video data compression system. 

BACKGROUND OF THE INVENTION 

In such a situation, i.e. when additional digital data have to be added before 
transmission to an existing coded bitstream, the simplest solution is to decode said 

15 bitstream before canying out said addition. The modified bitstream is then re-encoded 

and transmitted. Unfortunately, a full decoding can generally be considered as expensive, 
as it requires a decoder and an encoder. Moreover, re-encoding with re-estimated modes 
and vectors may introduce artefacts on the areas left untouched by the logo addition. 

A less expensive solution has then been proposed in the international patent 

20 application WO 99/51033 (PHF98546). Before describing this solution, some information is 

first recalled concerning conventional video decoders and encoders. 

A conventional video decoder such as shown in Rg.l includes a decoding 
channel 12, which comprises in cascade a variable length decoding circuit 1, an inverse 
quantizing circuit 2 and an inverse frequency transform circuit 3 (respectively : VLD, IQ, 

25 IDCT), and a motion compensation channel 14, which comprises a picture memory 4 

receiving the output signals of the decoder, a motion compensation circuit 5 (said 
compensation taking into account the output signals of this memory 4 and motion vectors 
V(n) received by the decoder) and an adder 6 of the output signals of circuits 3 and 5 
(respectively : MEM, COMP, A). The output picture of the decoder (also sent to the 

30 picture memory 4) is reconstructed by adding the prediction (output of circuit 5) to the 

decoded residual signal available at the output of the circuit 3 of the decoding channel 
12. 

A conventional video encoder such as shown in Rg.2 includes an encoding 
and decoding channel 13, which comprises a discrete cosine transform circuit 25, a 
35 quantizing circuit 26, a variable length coding circuit 27 (respectively : DCT, Q, VLC) and, 





at the output of the circuit 26, in cascade, an inverse quantizing circuit 28 and an inverse 
discrete cosine transform circuit 29 (respectively : IQ and IDCT)/ and a prediction channel 
11, allowing to subtract a motion compensated prediction from the input signal and which 
comprises an adder 21, for the reconstruction of the pictures before prediction, a picture 
memory 22, a motion compensation circuit 23 and a subtracter 24 (respectively : 
A, MEM, COMP, S). Said compensation takes into account previously estimated motion 
vectors V(n). 



incoming bitstream in a transmission chain are illustrated in Fig.3. Said chain comprises a 
first encoder 31 (called "encoder 1"), a sub-system 305 for the addition of the logo to the 
coded bitstream available at the output of the encoder 31, and, after transmission, a 
decoder 35 ("decoder 2"). The sub-system provided between said encoder and decoder 
itself comprises, as illustrated, a decoder 32 ("decoder 1"), a logo adder 33 and an 
encoder 34 ("encoder 2 n ). Then, starting from said decoder 32 and said encoder 34 and 
taking advantage of their complementarity, some simplifications are made, in order to 
finally reach the outline of the transcoder according to the cited document. 



where the index (,1) in R(.) and P(.) designates the "first" encoder 31 (= encoder 1), I(n) 
is the original video input of the encoder, P(I'(n-l), 1 ; V(n)) is the prediction signal 
computed by applying motion vector V(n) to the previously "decoded" picture I # (n-1) and 
which has to be subtracted from the original input bitstream I(n) in order to obtain the 
residual signal R(n) to be coded, and R(n) designates this residual signal. It must be 
noted that the signal R'(n) available at the input of the prediction channel of the encoder 
differs from R(n) by a value e(n) called the coding error ; R'(n) being equal to R(n) + 
e(n), the signal at the output of the adder of this prediction channel is therefore I'(n) = 
I(n) + e(n). 

For the first decoder 32 that follows the encoder 31, one has similarly, 
according to Fig.l : 



where the index (,1) in I'(.), R'(.) and P(.) designates the ^rst" decoder 32, R'(n) is the 
decoded residual signal, P(I'(n-l), 1 ; V(n)) is the prediction to be added to R'(n), and 
I'(n) designates the output of the decoder. As seen above, I'(n,l) may also be written in 
the form : 



that is to say, the output of the decoder is the sum of the original input signal I(n) and of 
the coding error e(n) during the coding operation of I(n). At the output of the logo adder 
33, one has consequently : 



A possible method and a device allowing to add data such as a logo to an 



According to Hg-2, one has indeed, for the first encoder 31 : 
R(n,l) = I(n) - P(I'(n-l), 1 ; V(n)) (1) 



I'(n,l) « R'(n,l) + P(r(n-1), 1 ; V(n)) 



(2) 



I'(n,l) = I(n) + e(n,l) 



JXM)=I'(n,l)+Logo(n) (3) 
where I'(n) Is the output of the decoder 32 and Logo(n) the data (for instance, a fogo) to 
be added to the main bitstream. The resulting output J'(n) is sent towards the encoder 
34. 

5 For this second encoder 34, one has (similarly to the previous case of the 

first encoder 31) : 

R(n,2) = J'(n,l) - PCr(n-l), 2 ; V(n)) (4) 
where the index (,2) in R(.) and P(.) now designates this "second" encoder 34, 
P(J'(n-l),2 ; V(n)) is the prediction which has to be subtracted from the output J r (n-1) of 
10 the logo adder 33 in order to obtain the residual signal that has to be coded, and R(n) 

designates said residual signal. 

Finally, for the second decoder 35, one has (similarly to the previous case 
of the first decoder 32) ; 

J'(n,2) = R'(n,2) + P(J'(n-l), 2 ; V(n)) (5) 
15 where the index (,2) in J'(0, R'() and P(.) now designates this "second" decoder 35, 

R'(n) is the decoded residual signal, P(J'(n-l), 2 ; V(n)) is the prediction to be added to 
R'(n), and J'(n) designates the output of the decoder. As for the decoder 32, J'(n,2) may 
be also be written in the form : 
J'(n,2)=J'(n,l) + e(n / 2) 
20 J'(n,2) « I'(n,l) + Logo(n) + e(n,2) 

J'(n,2) = I(n) + e(n,l) + Logo(n) + e(n,2) (6) 
which means that the output signal J'(n,2) of the transmission chain is indeed equal to 
the sum of the original input signal I(n), of the first coding error (coding/decoding in the 
first encoder and decoder), of the second coding error (coding/decoding in the second 
25 encoder and decoder), and of the additional data. 

Then, using the linearity of the motion compensation operator, it may be 

written : 

P(J'(n-l), 2 ; V(n)) = P[((J'(n-l), 1) + (e(n-l), 2)) ; V(n)] (7) 
which allows to write the relation (4) in the form : 
30 R(n,2) = J'(n,l) - P[((J'(n-l), 1) + (e(n-l), 2)) ; V(n)] (8) 

Using again the linearity of the compensation operator, one has : 

P(J'(n-l), l ; V(n)) = P[(I'(n-l) + Logo(n-1)), 1 ; V(n)] 

or : 

P(J'(n-l), 1 ; V(n)> = P(I'(n-l), 1 ; V(n)) + P(Logo(n-l) / 1 ; V(n)) (9) 
35 Therefore the relation (8) becomes : 

R(n,2) = F(n,l) + Logo(n) - P(e(n-1),2 ; V(n)) 

- P(I'(n-l), 1 ; V(n)) - P(Logo(n-l) / 1 ; V(n)) (10) 
or, from the relation (2) : 



R(n,2) - R'(n-1) - P(e(n-1), 2 ; V(n)) 
+ Logo(n) - P(e(n-l) f 2 ; V(n)) 

- P(I'(n-l), 1 ; V(n)) - P(Logo(n-l), 1 ; V(n)) (11) 
which finally leads to the following relation (12) : 
R(n,2) - R'(n-1) - P(e(n-1), 2 ; V(n)) 

+ Logo(n) - P(Logo(n-l) ; V(n)) (12) 
which is the final system equation of the transcoder with logo adder as proposed in the 
cited document. 

The general outline of a transcoder without logo adder is first recalled in 
Fig. 4. It comprises a residue decoding branch 41 (variable length decoding VLD + inverse 
quantization IQ + inverse discrete cosine transform IDCT), an encoding and decoding 
branch 42 (discrete cosine transform DCT + quantization Q + variable length coding VLC 
; inverse quantization IQ + inverse discrete cosine transform IDCT), and an intermediary 
branch called a pseudo-prediction branch 43 (first subtracter S + memory MEM + motion 
compensation COMP on the basis of motion vectors V(n) + second subtracter S). This 
branch 43 is so called because it is not exactly a classical prediction as in a basic encoder, 
the first adder being here replaced by a subtracter- The signals R'(n,l), R(n,2), R'(n,2), 
e(n / 2) / V(n), P(e(n-1), 2 ; V(n)) previously cited are shown in Fig.4. 

The corresponding scheme of the transcoder according to the cited 
document -i.e. with a logo adder- is then illustrated in Fig.5 where, in comparison with 
Fig.4, the identical parts are designated in the same manner. The additional part is a logo 
addition branch 50 that includes a memory MEM 51, receiving the logo to be added 
(signal Logo(n)), a motion compensation circuit COMP 52, receiving the output of the 
memory 51 and the vectors V(n) and delivering a predicted data stream, a subtracter S 
53, for delivering the difference between the original signal Logo(n) and the motion 
compensated one P(Logo(n-l) ; V(n)) available at the output of the circuit 52 (predicted 
data stream), and an adder 54, for introducing the output signal of said subtracter 53 into 
the main bitstream (corresponding to the complete sequence of successive pictures). The 
logo addition is therefore implemented by means of a residue addition to the incoming 
bitstream, and this residue is formed by subtracting logo with a motion compensated logo 
prediction that is based on reference pictures containing logo previously stored and that 
uses the same modes and vectors as the main incoming bitstream. 

The method and device thus described (Fig.5) are an improvement with 
respect to the conventional solution in which the input stream is completely decoded, an 
addition of data is performed in the pixel domain, and the resulting video sequence is re- 
encoded (Fig.3). However, in this scheme of Fig.5, two motion compensations are 
performed : a first one, well known and provided for correcting the quality drift 
introduced by the quantization function, and a second one on the data to be added, in 



order to be able to keep, in the output stream, the motion characteristics (motion types, 
motion vectors,...). of the input stream. With these two motion compensation operations 
and the three OCT or IDCT blocks, this solution remains still complex. 

SUMMARY OF THE INVENTION 

It is therefore a first object of the invention to propose, with respect to the 
solution previously described for adding data to a previously existing coded bitstream, a 
further improvement, leading to a reduced complexity. 

To this end, the invention relates to a method of modifying data in an 
encoded data stream corresponding to successive pictures, which method comprises a 
decoding step of the input digital signals which are associated with each current picture, 
a re-encoding step, and, between said decoding and re-encoding steps, a prediction step 
comprising in cascade at least : 

(a) a first subtracting sub-step provided for determining an encoding error 
during said re-encoding step ; 

(b) a motion compensation sub-step between said current picture and a 
previous picture ; 

(c) a second subtracting sub-step between the decoded signals obtained 
after said decoding step and the motion-compensated signals obtained after said motion 
compensation sub-step, the output of said second subtracting sub-step corresponding to 
the input of said re-encoding step ; 

characterized in that it also comprises : 

(d) between said decoding step and said second subtracting sub-step, a first 
adding sub-step ; 

(e) between said first subtracting and motion compensation sub-steps, a 
second adding sub-step ; 

said modifying data being simultaneously introduced in the decoded input signal by 
means of said first adding sub-step and before motion compensation by means of said 
second adding sub-step. 

According to another embodiment, the invention relates to a method of 
modifying data in an encoded data stream to successive pictures, which method 
comprises a decoding step of the input digital signals which are associated with each 
current picture, a re-encoding step, and, between said decoding and re-encoding steps, a 
prediction step comprising in cascade at least : 

(a) a first subtracting sub-step provided for determining an encoding error 
during said encoding step ; 

(b) a first sub-step for converting frequential signals into spatial signals ; 



(c) a motion compensation sub-step between the current picture and a 
previous picture ; 

(d) a second sub-step for converting spatial signals into frequential signals ; 

(e) a second subtracting sub-step between the decoded signals obtained 
after said decoding step and the signals obtained after said second converting sub-step, 
the output of said second subtracting sub-step corresponding to the input of said 
re-encoding step ; 

characterized in that it also comprises : 

(f) between said decoding step and said second subtracting sub-step, a first 
adding sub-step ; 

(g) between said first subtracting sub-step and said first converting sub- 
step, a second adding sub-step ; 

said modifying data being simultaneously introduced in the decoded input signal by 
means of said first adding sub-step and before said first converting sub-step by means of 
said second adding sub-step. 

Another object of the invention is to still simplify said method, especially for 
consumer applications. 

To this end, the invention relates to a method of modifying data in an 
encoded data stream corresponding to successive pictures, which method comprises a 
decoding step of the input digital signals which are associated with each current picture, 
and a re-encoding step, 

characterized in that it also comprises an addition step for defining : 

(i) on the basis of these additional data and motion vectors previously 
defined, a predicted signal ; 

(ii) the difference between said additional data and said predicted signal, in 
view of an addition of said difference into said data stream between decoding and re- 
encoding parts. 

Another object of the invention is to propose video processing devices for 
adding data to a video coded data stream according to anyone of these three 
implementations of the method according to the invention. 

BRIEF DESCRIPTION OF THE DRAWINGS 

These and other aspects of the invention will be apparent from and 
elucidated with reference to the embodiment described hereinafter. 
In the drawings : 

- Rgs.l and 2 show conventional video decoder and encoder ; 

- Fig.3 shows a possible scheme of a transmission chain for adding a logo to 
an incoming bitstream ; 



— 



- Rg.4 illustrates the outline of a known transcoder, without logo adder ; 

- Fig. 5 illustrate such a transcoder when it is, according to the solution 
described in the previously dted document, provided with a logo adder ; 

- Rgs.6 to 8 illustrate three embodiments of the technical solution according 
5 to the present invention. 



DESCRIPTION OF THE INVENTION 

Since the two motion compensation operations in Rg.5 use the same motion 
vectors (i.e. the motion vectors of the input stream), and since a motion compensation 
10 can be considered as linear, it is possible to merge them, which leads to the simplified 

scheme depicted in Rg.6. In this first embodiment of the invention, three IDCT or DCT 
are applied on all the 8 x 8 pixels blocks of the input pictures. 

Since the DCT and IDCT are also linear, and since applying an IDCT to a 
signal that has passed through a DCT equals a unity transfer, the data adder can be 
15 simplified to obtain the scheme depicted in Fig.7. In this second embodiment of the 

invention, there are still three IDCT or DCT, but the DCT applied on the data is only 
performed where data have to be added : for example, if a logo is inserted, then the DCT 
is performed only on the picture area where the logo is put 

These two simplified schemes of Figs.6 and 7 are equivalent, in terms of 
20 picture quality, to Rg.5. A third embodiment of the invention may then be proposed, in 

which it is decided not to connect the quality drift that occurs because of the quantization 
function, which leads to the scheme depicted in Rg.8. In this scheme, the addition of the 
data is performed in the OCT domain (and not in the pixel domain) and the coding loop 
of Rg.5 (IQ->IDCT->MEM->COMP->DCT) has been suppressed. This solution, which 
25 introduces some quality drift compared to solutions of Rgs.6 and 7, is however of high 

interest since it requires far less processing : EXT/IDCT blocks and motion compensation 
blocks have been suppressed. 
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CLAIMS : 

1. A method of modifying data in an encoded data stream corresponding to 
successive pictures, which method comprises a decoding step of the input digital signals 
which are associated with each current picture, a re-encoding step, and, between said 

5 decoding and re-encoding steps, a prediction step comprising in cascade at least : 

(a) a first subtracting sub-step provided for determining an encoding error 
during said re-encoding step ; 

(b) a motion compensation sub-step between said current picture and a 
previous picture ; 

10 (c) a second subtracting sub-step between the decoded signals obtained 

after said decoding step and the motion-compensated signals obtained after said motion 
compensation sub-step, the output Qf said second subtracting sub-step corresponding to 
the input of said re-encoding step ; 
characterized in that it also comprises : 

15 (d) between said decoding step and said second subtracting sub-step, a first 

adding sub-step ; 

(e) between said first subtracting and motion compensation sub-steps, a 
second adding sub-step ; 

said modifying data being simultaneously introduced in the decoded input signal by 
20 means of said first adding sub-step and before motion compensation by means of said 

second adding sub-step. 

2. A method of modifying data in an encoded data stream to successive 
pictures, which method comprises a decoding step of the input digital signals which are 
associated with each current picture, a re-encoding step, and between said decoding and 

25 re-encoding steps, a prediction step comprising in cascade at least : 

(a) a first subtracting sub-step provided for determining an encoding error 
during said re-encoding step ; 

(b) a first sub-step for converting frequential signals into spatial signals ; 

(c) a motion compensation sub-step between the current picture and a 
30 previous picture ; 

(d) a second sub-step for converting spatial signals into frequential signals ; 

(e) a second subtracting sub-step between the decoded signals obtained 
after said decoding step and the signals obtained after said second converting sub-step, 
the output of said second subtracting sub-step corresponding to the input of said re- 

35 encoding step ; 

characterized in that it also comprises : 

(0 between said decoding step and said second subtracting sub-step, a first 
adding sub-step ; 
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(g) between said first subtracting sub-step and said first converting sub- 
step, a second adding sub-step ; 

said modifying data being simultaneously introduced in the decoded input signal by 
means of said first adding sub-step and before said first converting sub-step by means of 
5 said second adding sub-step. 

3. A method of modifying data in an encoded data stream corresponding to 

successive pictures, which method comprises a decoding step of the input digital signals 
which are associated with each current picture, and a re-encoding step, 
characterized in that it also comprises an addition step for defining : 
10 (i) on the basis of these additional data and motion vectors previously 

defined, a predicted signal ; 

(ii) the difference between said additional data and said predicted signal, in 
view of an addition of said difference into said data stream between decoding and re- 
encoding parts. 

15 4. A video processing device for adding data to a video coded data stream, 

carrying out the method according to anyone of claims 1 to 3. 
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Abstract 

The invention relates to a method of modifying data in an encoded data stream 
corresponding to successive pictures, which method comprises a decoding step of the 
input digital signals which are associated with each current picture, followed by an 
encoding step. According to a main embodiment of the invention, the modifying data are 
simultaneously introduced in the decoded input signal by means of a first adding sub-step 
and in a prediction loop by means of a second adding sub-step. 
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